Skip to content

Add qwen3.6-27b benchmark to submission checker#2629

Closed
pgmpablo157321 wants to merge 7 commits into
masterfrom
qwen3.6-27b-checker
Closed

Add qwen3.6-27b benchmark to submission checker#2629
pgmpablo157321 wants to merge 7 commits into
masterfrom
qwen3.6-27b-checker

Conversation

@pgmpablo157321

Copy link
Copy Markdown
Contributor

Partially address #2628

@pgmpablo157321
pgmpablo157321 requested review from a team as code owners July 13, 2026 23:37
@github-actions

Copy link
Copy Markdown
Contributor

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

pgmpablo157321 and others added 4 commits July 13, 2026 22:45
This reverts commit 7d8ae1a.
…tale results.json (#2631)

The endpoints accuracy artifact was renamed from results.json (with a
responses field) to accuracy/accuracy_results.json in
mlcommons/endpoints#400, which also dropped the responses field
entirely (per-sample text now lives in events.jsonl, not part of the
submission). truncate_accuracy_log.py still looked for the old
results.json/responses shape and would only log "missing" for every
endpoints submission. Detect the endpoints layout via the scenario-root
config.yaml and skip truncation, and remove the now-dead
PERFORMANCE_ENDPOINTS_DIR/ACCURACY_ENDPOINTS_DIR constants left over
from the loader.py refactor to ENDPOINTS_SCENARIO_DIR.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants